Combining performance and outcome indicators can be used in a standardized way: a pilot study of two multidisciplinary, full-scale major aircraft exercises.

Radestad et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 201 2, 20;58
http://www.sjtrem.eom/content/20/1/58

SCANDINAVIAN JOURNAL OF

trauma, resuscitation
£t emergency medicine

ORIGINAL RESEARCH Open Access

Combining performance and outcome indicators
can be used in a standardized way: a pilot study
of two multidisciplinary, full-scale major aircraft
exercises

Monica Radestad^", Helene Nilsson^, Maaret Castren\ Leif Svensson^, Anders Ruter"^ and Dan Gryth^

Abstract

Background: Disaster medicine is a fairly young scientific discipline and there is a need for the development of
new methods for evaluation and research. This includes full-scale disaster exercisers. A standardized concept on
how to evaluate these exercises, could lead to easier identification of pitfalls caused by system-errors in the
organization. The aim of this study was to demonstrate the feasibility of using a combination of performance and
outcome indicators so that results can be compared in standardized full-scale exercises.

Methods: Two multidisciplinary, full-scale exercises were studied in 2008 and 2010. The panorama had the same
setup. Sets of performance indicators combined with indicators for unfavorable patient outcome were recorded in
predesigned templates. Evaluators, all trained in a standardized way at a national disaster medicine centre, scored
the results on predetermined locations; at the scene, at hospital and at the regional command and control.

Results: All data regarding the performance indicators of the participants during the exercises were obtained as
well as all data regarding indicators for patient outcome. Both exercises could therefore be compared regarding
performance (processes) as well as outcome indicators. The data from the performance indicators during the
exercises showed higher scores for the prehospital command in the second exercise 15 points and 3 points
respectively. Results from the outcome indicators, patient survival and patient complications, demonstrated a higher
number of preventable deaths and a lower number of preventable complications in the exercise 2010. In the
exercise 2008 the number of preventable deaths was lower and the number of preventable complications
was higher.

Conclusions: Standardized multidisciplinary, full-scale exercises in different settings can be conducted and
evaluated with performance indicators combined with outcome indicators enabling results from exercises to be
compared. If exercises are performed in a standardized way, results may serve as a basis for lessons learned. Future
use of the same concept using the combination of performance indicators and patient outcome indicators may
demonstrate new and important evidence that could lead to new and better knowledge that also may be applied
during real incidents.

Keywords: Airplane crash. Disaster preparedness. Disaster management. Disaster response. Educational model.
Field exercise. Major incident. Simulation

* Correspondence: monica.radestad{S)ki.se

'Karolinska Institutet, Department of Clinical Science and Education and
Section of Emergency Medicine, Sodersjukhuset, Stockholm, Sweden
Eull list of author information is available at the end of the article

O© 2012 Radestad et at; licensee BioMed Central ttd. This is an Open Access article distributed under the terms of the Creative
BIOIVIGCI CCntrBl commons Attribution ticense (httpy/creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and
reproduction in any medium, provided the original work is properly cited.

Radestad et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 201 2, 20;58
http://www.sjtrem.eom/content/20/1/58

Page 2 of 8

Background

The overall aim for medical responses to disasters and
major incidents (Mis) is to achieve the best possible out-
come for the most number of victims. However, in order
to evaluate the outcome of a response you also need to
assess the processes involved. To be able to do this there
is a need for standardization and for sets of goals
(bench-marks) reflecting what is to be considered as
good and less good performance. These processes and
goals must be developed in a way that allows them to be
systematically studied, analyzed and with the possibility
to compare results and experiences [1-3]. Several
researchers have indicated the need for validated assess-
ment methods performed in a scientifically genuine
manner in order to measure the effectiveness of disaster
medicine response [4-8].

In most healthcare systems the use of quality indica-
tors are mandatory for determining standards and meas-
uring quality. The challenge is to address indicators even
in the field of disaster medicine. The use of measurable
quality indicators to determine the level of performance
against predetermined goals and objectives has been
addressed in previous studies [9-11]. In these studies the
quality indicators represent standards (times and con-
tent) of what is desirable performance of the manage-
ment during Mis and disasters and were results from
process- and concept modelling and/or opinion of an
expert panel [10].

A few educational models for simulation of disaster
management and Mis response are accepted and used in
Sweden [12,13]. However, there are a limited number of
full-scale exercises (FSE) and systematic and validated
evaluation tools are still in the process of being devel-
oped. Therefore the competence and effectiveness of the
emergency medical services (EMS) and hospital per-
formance seldom are assessed in a way that allows
results to be compared. The question arises whether a
standardized evaluation methodology would be applic-
able in a FSE to assess if and how the management is
related to the patient outcome. The aim of this study
was to demonstrate the feasibility of using a combin-
ation of performance and outcome indicators so that
results can be compared in standardized full-scale
exercises.

Methods

Description of the exercises

A quantitative evaluation method was applied in two
multidisciplinary, FSE (major aircraft accident) in two
regions in Sweden. The two exercises studied were con-
ducted in October 2008 and in April 2010 at two differ-
ent airports. The time interval, 18 months, between the
exercises is due to the willingness of the two different
airports and county councils to participate in this study.

According to national regulations, each international air-
port is mandated to perform an exercise every other
year, and there were, to our knowledge, no other airport
exercises during this time interval [14].

Exercise scenario

The scenario was: an aircraft carrying passengers and
crew that crashed during a landing attempt resulting in
99 respectively 100 victims (99 for the reason that one
person, aimed to be figurant, felt sick on a late notice).
In both exercises the participants, EMS and hospital
personnel, were all familiar with the management struc-
ture (doctrine) at MI and disaster and expected to re-
spond according to plans and procedures as stated in
the national regulations issued by the Swedish National
Board of Health and Welfare [15].

A total of 131 and 69 health care workers, respectively
participated in the exercises. The exercises were con-
ducted in real-time. All the participants were alerted and
dispatched according to the disaster plan. The available
resources in both exercise settings, ambulance, helicop-
ter, health care, fire and rescue service, police and other
responding agencies, were all defined in beforehand. Fig-
urants acted as mock victims and had injuries that were
appropriate to what was to be expected in this type of
accident. Each victim had their injuries visualized on a
figurant- card and all injuries had predetermined med-
ical needs according to a specific template in the Emergo
Train System® (ETS) victim bank [12].

The victim's conditions were expressed in physio-
logical parameters. The basis for the victim bank has
been developed on a national consensus among trauma-
tologists. ETS is a simulation tool for education and
training in disaster medicine which can be used both for
table top exercises and field exercises as well as tem-
plates for evaluating results of performance and the
treatment of the victims. A similar system is the Mass
Casualty Simulation system (MACSIM) but with further
developed injury card [13].

Exercise design and mock victim distribution

Figure 1 shows how the exercise design was built up
using the flexibility of a standardized victim set. The
flow chart of victims is based on distribution from the
scene of the accident through virtual transport to the
hospital emergency department (ED). All victims were
on-site triaged and treated by medical personnel and dis-
tributed to hospitals by the designated duty officer
(DDO) at the strategic level of management (Figure 2).
In 2008 live figurants were used to act as victims at the
ED and in 2010 magnet victim symbols were used and
displayed on whiteboards according to the simulation
system [12].

Radestad et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 201 2, 20;58
http://www.sjtrem.eom/content/20/1/58

Page 3 of 8

Local level on Scene*
i

]

Regional (strategic) level*

*

Local level at Hospital"*
i

ED

OR, ICU, XRAY

Live figurants
with figurant-
cards 2008

Virtual

■=>

transport

Magn et
victim
symbols
displayed
on wh iteboards
2010

Live figurants
wjith figurant-
cards 2008

Magn et
victim
symbols
displayed
on wh iteboards
2010

Figure 1 Flow chart of victim distribution. Overall design of two full-scale disaster exercises including the possibility to use figurant-cards,
combined with tabletop simulation. *At the regional level results from performance indicators were obtained. **At the local level on scene and in
hospital results from both performance indicators and patient outcome were obtained.

Injury panorama Aircraft accident

Priority

Unconscious > 10 min "brain contusion"
Traumatic brain injury

Thoracic injuries (haemothorax-pneuraothorax)

Pulmonary contusion

Blunt abdominal trauma/Major pelvic rupture with clinical signs of
shock

Penetrating abdominal injury

Concussion

Neck trauma- suspected spinal injury
Fracture- need of operation

Lacerations/hand injury- need of operation

Thoracolumbar injuries

Minor injuries- assessment in the Emergency room

Severe psychological shock

No visible injury

Severe multiple injury or bum injury 90% of a Body Surface Aiea

Dies on scene before transport

27%
Tl (red)

22%
T2
(yellow)

39%
T3
(green)

I

2%

10%

Figure 2 Distribution of predetermined priorities of victims (n = 100) according to the injury panorama in ETS. Triage categories were
based on the physiological parameters obtained from the figurant-card as respiratory rate, pulse rate, systolic blood pressure and Glasgow Coma
Scale. Tl (red) immediate, severely injured; T2 (yellow) urgent, moderately injured; T3 (green) not urgent, minor injured; (Black) dead.

Radestad et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 201 2, 20;58
http://www.sjtrem.eom/content/20/1/58

Page 4 of 8

Table 1 Sets of performance indicators and standards
used for evaluation in full-scale disaster exercises

Prehospital

command and control

Standard

(time frame in min)

Putting on tabard*

First report to dispatcli

Content on first report

Formulate guidelines for response

Establish contract with
strategic level of command

Liaison with fire and police

Second report from scene

Content of second report

Establish level of medical ambition
First patient evacuated
nformation to media on scene
Regional command and control

Declaring major incident

Deciding level of preparedness

Decision on additional resources
to scene

Deciding on receiving hospitals

Establishing contact with incident
officers at scene

Deciding on guidelines for referring
hospitals

Brief information to media

Formulate general guidelines in
accordance with guidelines from scene

Make sure there is information for
definitive referral guidelines

Evaluated if capacity of own
organisation is sufficient

Notify guidelines on referring
hospitals

Hospital command and control

Decide on level of preparedness

Formulate guidelines for hospital
response

nform media

Give information about resources
to strategic level

Ensuring that there is a medica
officer in emergency operation

Estimate need of ICU beds

First information to staff

Estimate endurance of staff

Directly*
2

IVl ETHANE**

3

5

5

10

Verifying first report
Indicating first
patient transport

10

15

30

Standard

(time frame in min)
1

3
3

10

10

15
15

20

30

40

Standard

(time frame in min)

3

15

15
25

30

45

60
90

Table 1 Sets of performance indicators and standards
used for evaluation in full-scale disaster exercises

(Continued)

Evaluate and report estimated
shortage of own capacity

Evaluate influence on the daily
hospital activities

Information plan for patients with
postponed appointments and operations

Regional and Hospital staff procedure skills

120

120

Assigning functions to all the staff
members directly upon arrival

Placement in room according to
function in staff

Designated telephone numbers

Introduction of newly arrived staff
member

Utilization of available equipment^

Maximum 8 min for "staff briefing"

Content of "staff briefing"''

Telephone discipline during "staff
briefing"

Drawing and content of "staff
schedule"

Summary after session, orally
Summary after session, written

Standard
(time frame in min)

Directly on arrival

Directly

Directly
Maximum 1

Yes/No

Yes/No

Yes/No
Yes/No

* Vest, clearly labelled for identification of medical and ambulance staff.
** Major incident declared. Exact location. Type of incident. Hazards,
Accessibility, Number of casualties. Emergency Services required. Acronym for
the content, defined in the IVlajor Incident Medical IVlanagement and Support
Course (IVIIMMS).

^ Equipment available; v\/hiteboard, flipchart, fax, and computer.

^Reports from all functions, summarizing, assigning new tasks, time for next

briefing.

Instead of actually transporting the victims to the hos-
pital by ambulances, ambulances were in both settings
used only at the scene of the accident, and for a short
transportation to the point where victims data were
reported to the receiving hospitals. This data, regarding
measures performed on each victim, was at this point
collected by trained nurses and reported by telephone to
the hospital ED. Also, the expected time of arrival
(according to real transportation time) was reported.
This meant, that a mock-victim in 2008 and magnet vic-
tim symbol in 2010, were presented with exactly the
same data as the mock-victim that had participated on-
scene, could be presented at the ED. By using this
method, fewer ambulances were needed at the scene of
the accident although the realism was the same. In a real
MI probably 25-30 ambulances would rapidly respond
to the requirement through the dispatch centre. For ob-
vious reason access to ambulances is limited in FSE.
However, for testing the concept merely two hospitals
participated, one in each exercise, and received 17

Radestad et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 201 2, 20;58
http://www.sjtrem.eom/content/20/1/58

Page 5 of 8

victims, respectively. Both hospitals were emergency
hospitals, and had a capacity of 302 and 309 beds, re-
spectively. Available intensive care unit beds were 13
and 12 respectively, and operating rooms were nine and
ten respectively. The time period in hospital, for each
victim, included the time spent at the ED, x-ray, surgery
(OR) and the estimated time for post operative- and in-
tensive care. Performance- and patient outcome indica-
tors were included in the evaluation of both exercises.

Data collected

The external evaluators were registered nurses and
doctors certified as instructors in a Swedish national
concept in disaster preparedness [15]. One or two eva-
luators were positioned at designated areas and used a
protocol with sets of performance indicators as tem-
plates, all trained in a standardized way at a national
disaster medicine centre (Table 1). These indicators
were the same as those that had been used in previous
studies [9-11].

The protocols were used for evaluation of performance
at two different levels of medical management, regional
level (in international literature often called strategic or
gold level) and local level i.e. on-scene and in hospital
(Figure 1).

The performance indicators covered early decision-
making (management skills), as well as staff perform-
ance. The level was scored by giving 0, 1, or 2 points
where 0 point was given if the performance was not
timely or adequate, 1 point if the result was somewhat
correct and 2 points for a timely and by content, correct
performance. Each template contained 11 indicators,
thereby allowing for 22 points being maximum achiev-
able level of result (Table 1). Eleven out of 22 points
where considered satisfactory.

The outcome indicator, patient outcome, was evaluated
in terms of risk for preventable death or complication
[12]. Each victim was assigned to specific measures in
order to be expected to have a favorable outcome, all
according to templates that are included in the ETS. If
these treatments were not performed or performed too
late, according to the ETS stipulated timeframe, the pa-
tient will have a preventable complication or death. In

ETS every specific treatment or transfer are performed
according to a "real-time" approach.

All logistical and clinical data were registered at the
scene of the accident and in hospital, regarding the care
performed in real time, by participants who had been
trained and instructed in ETS template.

Results

All data regarding the performance indicators of the
participants during the exercises were obtained as well
as all data regarding indicators for patient outcome.
Both exercises could therefore be compared regarding
performance (process) as well as outcome (result)
indicators.

Performance indicators

All result were on approved level except for prehospital
command in the first exercise 2008 that scored 3 points
compared to the second exercise 2010 that scored 15
points. Apart from this result, all other results were on
the same level in the two exercises (Table 2). In common
for the regional management groups is that there were
low scores for establishing contact with incident officers
at the scene, first information to media and estimating if
resources within their own organization were adequate
for managing the accident. All management groups
(hospital and strategic) received high score in perform-
ance of staff skills as well as regarding correct and timely
decisions.

Outcome indicators

Results from the patient outcome indicators, that is pre-
ventable death and/or preventable complications,
demonstrated a higher number of preventable deaths in
exercise 2010, (n = 7) compared in the 2008 exercise
(n = 5). Regarding the preventable complications the
results were opposite with a higher number in the 2008
exercise (n = 9) compared to the 2010 exercise (n = 5)
(Table 3). Out of 99 respectively 100 victims, the partici-
pating hospital in each exercise received totally 17
injured victims prioritized as Tl and T2. The numbers
of ambulances were according to plan and were consid-
ered to be on an adequate level (simulated as previously

Table 2 Results based on templates of performance indicators, expressed in points

Management level Exercise 2008 Exercise 2010

Category Category

Command & control* Staff procedure* skills Command & Control* Staff procedure* skills

Prehospital (Local level) 3 Not assessed 15 Not assessed

Regional 15 17 18 21

Hospital (Local level) 17 21 17 20

*Maximum score was 22 points in each category where 1 1 different indicators were given 0, 1 or 2 points. Score: Correct decision and in right time. Correct = 2
points, Partly correct = 1 points, Incorrect = 0 points.

Radestad et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 201 2, 20;58
http://www.sjtrem.eom/content/20/1/58

Page 6 of 8

Table 3 Patient outcome expressed as preventable
complications and preventable death in two, full-scale
disaster exercises

Exercise 2008

Exercise 2010

Preventable Complication

53% (9/1 7*)

29% (5/1 7*)

Preventable Death

29% (5/1 7*)

41% (7/17*)

*AII 17 victims receiving the participating hospital were at risk, according to
ETS template, for having unfavorable outcome expressed as preventable
complication or preventable death.

described). The first victim arrived at the ED 100 min
respectively 105 min after the accident, for the last vic-
tim this was 235 min and 273 min respectively.

Discussion

This study demonstrates the possibility of conducting
standardized exercises with built-in evaluation method-
ology that can produce comparable results. The discus-
sion in this paper focuses more on the methodology
than the results from the exercises per se. A prerequisite
for results from evaluations to be comparable is that the
exercises are evaluated the same way, making it possible
to obtain valid data to be recorded. In the present study
we have demonstrated the possibility to apply both per-
formance and outcome indicators in multidisciplinary,
FSE. Indicators should be based on well-developed stan-
dards established and accepted by the organization to
ensure comparability and reproducibility. The selected
indicators were derived from a national concept and
process modeling conducted by the National Board of
Health and Welfare in Sweden [15]. Idvall et al.
describes this as 'the key to good quality indicators' and
the only certain way of knowing what is good or less
good by comparing performance against the standard
[16]. Furthermore, Idvall et al. clarifies that the most im-
portant indicator is the patient outcome, and it is a
product influenced by all activities in an organization
and should be measured together with other indicators.
Standards (bench-marks) for evaluation methods that
examine and describe the relation between the perform-
ance in disaster management and the patient outcome
are yet to be described [17].

The evaluation model exposed several challenges faced
in the initial decision-making procedures that were re-
peatedly observed and had major impact on patient
outcome. For example, insufficient reporting and incon-
sistent coordination among responders that limited the
ability to evacuate severely injured victims and resulted
in comparatively high numbers of preventable death and
complications. Despite good access to available ambu-
lances in both exercises failure of rapid evacuation indi-
cates that decisions made in this context were less than
optimal which could possibly have been reflected in the
patient outcome. By using the proposed model each

agency will have increased possibilities to specifically
identify what part of the command and control that
needs to be improved. The fact that the difference be-
tween the prehospital command and control, as illu-
strated in Table 2, did not show in patient outcome can
however, not be explained and this is something that
may have to be addressed using different research meth-
ods. At this stage, after only a few exercises, it is still too
early to assess any relationship between performance
and outcome. There is also a need to study if timely
decisions or actions are more important than the con-
tent of the performance. In the present study the two
prehospital organizations had different training concepts
with regard to command and control, even though they
had to follow the same regulations in performing [15].

Efficient disaster management requires information
sharing, follow up and coordinated decision-making
among involved agencies. Jufferman and Bierens studied
five national disaster responses and noted that many
shortages are repeated despite changes in protocols, le-
gislation and organizations [6]. For this reason there is a
need for interagency training to improve operational
decisions. Circumstances at the scene such as time
-period, geography, weather conditions, reliable commu-
nication systems and security are other essential factors
that may affect the response and therefore have to be
considered in the assessment. This problem was recently
described after a real aircraft incident where there were
difficulties to evacuate the victims and this delayed the
distribution of victims to hospitals [18].

The use of standardized education and training mod-
els, with built-in evaluation possibilities, allows compari-
sons and could lead to better knowledge and improve
disaster medicine evidence in management system. In
this study the first exercise was reproduced and the
same type of evaluation, obtaining the same types of
comparable results, was used in the second exercise. To
be able to draw valid conclusions, the number of exer-
cises performed must be higher. But this is also depend-
ing on number of indicators and methodology chosen.
However, in our opinion, we must already start now to
collect data as described in this study in order for
the lessons learned in the processes not to be delayed
further. The rational for this is the development of a
robust design of the evaluation model used in the
present study.

In the planning phase consideration was, in this study,
taken both to each hospitals request of participation as
well as their concerns regarding financing of staff which
had impact upon the exercise design (Figure 1). In our
view, the difference in methodology regarding live figur-
ants vs. virtual victims used at hospital, shows the mod-
els robustness in several settings. It is our belief that
before engaging several hospitals in this type of

Radestad et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 201 2, 20;58
http://www.sjtrem.eom/content/20/1/58

Page 7 of 8

Structured exercises it should be prioritized to evaluate
the first steps of the process. When this model becomes
more accepted, additional units that can be trained and
evaluated should be introduced.

Significance of tliis study for tlie future

To achieve greater acceptance methods with valid indi-
cators need to be promoted by the health care providers.
By using the same indicators in the evaluation of real
incidents, in relation to available patient data, it would
be possible to not only compare the performance indica-
tors but also to validate the outcome indicators chosen
in the exercises. The comparison with real incidents will
give clues if we are educating and training the staff in
the best way, and comparing results will provide infor-
mation about possible improvements of the pedagogic
educational simulation system used today. Knowledge
that will emerge from this will no doubt, improve the
possibility to conduct measurements in order to improve
the disaster response before a real incident occurs. It will
also reduce the time consumption for planning and
after-action report writing.

Indicators are now gradually introduced in different
regions in Sweden, so far, for evaluating the effect of
training in order to measure quality in performance
related to the management doctrine, reflected by re-
gional regulations, supported by the Swedish National
Board of Health and Welfare. Also in an international
study results from using the same sets of indicators has
recently been published [19].

In this study we have not considered structure indica-
tors relating to staff, time and financial costs. Results
from future studies with a systematic approach to the
structure indicators will provide important information
on how train and exercises should be conducted in the
best way.

Limitations

Competency and quality between different levels of
management and organisations in this paper can depend
on variation of personnel's professional qualifications,
education, experience and training in disaster medicine
and might be a limitation. This applies also to the eva-
luators. Different evaluators may give different scoring
for different processes. There is also a need to study if
timely decisions or actions are more important than the
content of the performance.The used simulation system
may have limitations for evaluating patient outcome, but
until studies from real incidents (where performance
indicators as well as outcome indicators) are available
we believe that this method will provide valid results.

The basis to use 11 points as acceptable level for
measuring performance is based on results from training
and examination sessions in command and control

during several years of experience [9-11]. However, it
could well be that any of the indicators could be more
important than the others. Correlations studies may give
answer to this. It may also well be that the indicators in
the future must be weighted.

The figurant in the first exercise, that fell sick on late
notice did not simulate a critical injured patient and was
not intended to be sent to the participating hospital.
Therefore we do not believe that this has any effect on
the results.

Cost effectiveness, in regards to this subject, was
not addressed in this study. This is due to the lack
of systematic documentation of total costs for all
involved staff including time for planning as well as the
actual exercise.

Conclusions

Standardized FSE in different settings can be conducted
and evaluated with performance indicators combined
with outcome indicators enabling results from exercises
to be compared. If exercises are performed in a standar-
dized way, results may serve as a basis for lessons
learned. Future use of the same concept using the com-
bination of performance indicators and patient outcome
indicators may demonstrate new and important evidence
that could lead to new and better knowledge that also
may be applied during real incidents.

Competing interests

This work was performed by Stockholm Prehospital Centre (SPC) in
cooperation with the Centre of Teaching in Disaster Medicine and
Traumatology (KMC), Linkoping University, Sweden and the Sophiahemmet
University College, Stockholm, Sweden. The authors declare that they have
no competing interests.

Authors' contributions

MR was involved in the study design, exercise data collection, analysis, and
manuscript writing. HN was involved in the study design and data collection.
MC and IS contributed to the finalization of the manuscript AR and DG was
involved in the study design and took an active part in the data collection,
analysis and manuscript writing, revision and editing. All authors read and
approved the final version of the manuscript

Acl<nowledgments

This study was supported by grants from the Stockholm County Council. The
authors thank the Air Navigation Services of Sweden and the County
Administrative Board of Stockholm and the County Administrative Board of
Jamtland for the opportunity to participate in the exercises that resulted in
this study, as well as the support from different organizations that
contributed to this project

Author details

^Karolinska Institutet, Department of Clinical Science and Education and
Section of Emergency Medicine, Sodersjukhuset, Stockholm, Sweden. ^Centre
for Teaching and Research in Disaster Medicine and Traumatology, Linkoping
University, Linkoping, Sweden. ^Karolinska Institutet, Department of Clinical
Science and Education, Sodersjukhuset, Stockholm, Sweden. ^Sophiahemmet
University College, Stockholm, Sweden. ^Karolinska Institutet Department of
Physiology and Pharmacology and Section of Anaesthesiology and Intensive
care, Stockholm, Sweden.

Received: 19 March 2012 Accepted: 19 August 2012
Published: 28 August 2012

Radestad et al. Scandinavian Journal of Trauma, Resuscitation and Emergency Medicine 201 2, 20;58
http://www.sjtrem.eom/content/20/1/58

Page 8 of 8

References

1. Sundnes KO, Birnbaum ML: Health disaster management guidelines for
evaluation and research in the Utstein style. Prehosp Disaster Med 2003,
17(Suppl 3):1 13-127.

2. Lennquist S: Promotion disaster medicine to a scientific discipline - A
slow and painful but necessary process. Editorial. International Journal of
Disaster Medicine 2003, 2:95-96.

3. Auf Der Heide E: The importance of evidence-based disaster planning.
Ann Emerg Med 2006, 47:34-49.

4. Kaji AH, Lewis RJ: Assessment of the reliability of the Johns Hopkins/
Agency for Healthcare Research and Quality hospital disaster drill
evaluation tool. Ann Emerg Med 2008, 52:204-210.

5. Jenckes M, Catlett C, Hsu E, et ah Development of evaluation modules for
use in hospital drills. Am J Disaster Med 2007, 2:87-95.

6. Juffermans J, Bierens JJ: Recurrent medical response problems during five
recent disasters in the Netherlands. Prehosp Disaster Med 2010,
25(2):137-138.

7. Savoia E, Biddinger PD, Burstein J, Stoto MA: Inter-agency communication
and operations capabilities during a hospital functional exercise:
reliability and validity of a measurement tool. Prehosp Disaster Med 2010,
25(1):52-58.

8. Legemaate G, Burkle F, Bierens J: The evaluation of research methods
during disaster exercises: Applicability for improving disaster health
management. Prehosp Disaster Med 2012, 27(l):18-26.

9. Ruter A, Ortenwall P, Vikstrom T: Performance indicators for major
incident medical management - a possible tool for quality control?
Internatlonai Journal of Disaster Medicine 2004, 2:52-55.

10. Ruter A, Lundmark T, Odmansson E, Vikstrom T: The development of a
national doctrine for management of major incidents and disasters.
Scand J Trauma Resusc Emerg Med 2006, 14:189-194

11. Ruter A, Ortenwall P, Vikstrom T: Staff procedure skills in management
groups during exercises in disaster medicine. Prehosp Disaster Med 2007,
22{4):31 8-321.

12. Emergo Train System'^. 2012. Available at website
http7/ WA/vw.emergotrain.com Accessed 29 May.

1 3. Mass Casuaity Simulation system (MACSIM) ®. 201 2. Available at website
http7/ vmvw.macsim.se. Accessed 29 May.

14. Swedish transport agency. Regulation TESF 2010:1 14. 2012. Available at
website http://wwwtransportstyrelsen.se Accessed 16 February.

15. The Swedish National Board of Health and Welfare: Regulation SOSFS 2005.
13th edition. 2012. Available at website
http://www.socialstyrelsen.se/sosfs/2005-1 3 Accessed 24 August

16. Idvall F, Rooke L, Hamrin F: Quality indicators in clinical nursing: a review
of the literature. J Adv Nurs 1 997, 25:6-1 7.

17. Nilsson H, Ruter A: Management of resources at major incidents and
disasters in relation to patient outcome: a pilot study of an educational
model. Eur J Emerg Med 2008, 15:162-165.

18. Postma ILE Winkelhagen J, Bloemers FW, Heetveld MJ, Bijisma TS, Gosling
JC: February 2009 Airplane Crash at Amsterdam Schiphol Airport: An
Overview of Injuries and Patient Distribution. Prehosp Disaster Med 201 1,
26(4):299-304

19. France JM, Nichols D, Dong S: Increasing emergency medicine residents'
confidence in disaster management: use of an Emergency Department
simulator and an expedited curriculum. Prehosp Disaster Med 2012,
27(l):31-35.

doi:1 0.1 1 86/1 757-7241 -20-58

Cite this article as: Radestad et ai: Combining performance and
outcome indicators can be used in a standardized way: a pilot study of
two multidisciplinary, full-scale major aircraft exercises. Scandinavian
Journal of Trauma, Resuscitation and Emergency Medicine 2012 20:58.

Submit your next manuscript to BioMed Central
and take full advantage of:

• Convenient online submission

• Thorough peer review

• No space constraints or color figure charges

• Immediate publication on acceptance

• Inclusion in PubMed, CAS, Scopus and Google Scholar

• Research which is freely available for redistribution

Submit your manuscript at \ rant,,\

www.biomedcentral.com/submit Biomea eencrai